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Abstract 


This study presents Arizona’s innovations in academic accountability policy and academic 
accountability frameworks for alternative schools. A timeline of statutes and regulations 
including the State Board of Education approved alternative school definition provides Arizona’s 
context for alternative school accountability policy and frameworks. Arizona is relatively unique 
with its development of two state-level accountability frameworks for alternative schools. In 
addition to presentation of the Arizona Department of Education’s (ADE) Alternative A-F model 
and the Arizona State Board for Charter Schools (ASBCS) Academic Framework for Alternative 
Schools, this study offers a comparison of the two systems. Bar graphs present two years, fiscal 
years 2012 and 2013, of accountability results. Correlation coefficients for both years show a 
strong correlation between ADE’s Alternative A-F and ASBCS’ Academic Framework. This 
analysis provides empirical findings that will help further discussion of these innovations in 
policies and frameworks leading to more appropriate accountability for alternative education 
campuses. (Contains 14 figures and 4 tables) 
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Purpose and Objectives 

The purpose of this study is to present Arizona’s innovations in academic 
accountability models for alternative schools and findings from the first two years of 
academic achievement labeling resulting from these policy innovations and 
subsequent frameworks. This paper’s objectives are: 

1. Describe the context in the state for alternative school accountability policy and 
frameworks. 

2. Present the two accountability frameworks for alternative schools. 

3. Present two years of accountability results for the state’s alternative schools. 

4. Begin discussion of whether these policy innovations bring more appropriate 
academic labels for alternative schools that educate high-risk students. 

Perspectives and Theoretical Framework 

Perspectives 

Research and scholarly publications on the influence of race-ethnicity, urban 
settings, and socio-economic status on the achievement gap or students “at-risk” 
abounds. In contrast, inquiry into alternative education is relatively limited. 
Alternative education research is often linked to the special needs of the student 
population served by alternative schools (Gorney & Ysseldyke, 1993; Kattsiyannis 
& Williams, 1998; Lehr & Lange, 2003). In 2009, the Alternative Schools 
Research Project published a synthesis of state-level policy on alternative schools 
(Lehr, Tan, & Yssekdyke). Much has happened in public education since that 


report used 2008 data. 
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In 2010, Jobs for the Future issued its report. Reinventing Alternative 
Education: An Assessment of Current State Policy and How to Improve It. In that 
work, one of the recommendations is that states adopt an accountability framework for 
alternative schools (Almeida, Le, Steinberg, & Cervantes, 2010). A comparison of 
the alternative education accountability policies and frameworks in the six states 
identified in the Jobs for the Future report provides more detail about those six states 
(Schlessman & Hurtado, 2012). A 2013 issue of the Educational Researcher features 
an essay about rethinking the common assumptions about the relationship among the 
federal government, states, and local agencies in policy making for public education 
(Marsh & Wohlstetter). 

Arizona began revising its school achievement profiles in 201 1 and its 
alternative school accountability model in School Year 2011 - 2012. There are some 
innovative elements of the Arizona Department of Education’s alternative school 
accountability model such as its Improvement measure and Academic Persistence 
(Arportela & Laczko-Kerr, 2013). The Arizona State Board for Charter Schools 
Academic Framework for Alternative Schools (2014) offers even more innovation to 
alternative school accountability by identifying subgroups, i.e. English Language 
Learners, Free and Reduced Lunch eligible students, and students with special needs, 
and appropriate minimum subgroup sizes. However, school leaders are often 
confused about the similarities and differences between the two accountabilities. 

There has not been an effort to compare and contrast the two frameworks. This paper 
presents a comparison of the two accountabilities which operationalize policy 


decisions. 
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Theoretical Framework 

The author of this study recognizes the “bias” that humans bring to empirical 
inquiry. State-level policy makers chose to use quantitative instruments, e.g. the state- 
mandated test of standards, state reports of cohort status and graduation rate, etc. in 
the alternative school academic accountability frameworks. Other policy analysis 
looks at the legal and compliance interpretation that state and school level educational 
leaders use when implementing policy (Hemmer, Madsen, & Torres, 2013). Legal 
and compliance interpretation is beyond the scope of this inquiry. 

This study does recognize that policy decision is a directive that is 
operationalized in an instrument. Scholarly thinking, dating back to Kant’s 
epistemology (Russell, 1945) and American pragmatism such as Peirce (Buchler, 
1955) and Dewey, has identified limits that instrumentation brings to inquiry. Some 
of the instruments used in the Arizona frameworks, specifically the way that full 
academic year is defined for alternative schools or graduation rate as seen through the 
lens of cohort status, is subject to data integrity issues such as human error and the 
limitations of the state’s student data reporting system. The limits of the selected 
instruments are a limitation reflected in the data presented in this study. 

Even though much of this initial analysis remains at the descriptive level, this 
inquiry satisfies some minimum canons of logic, e.g., no generic fallacy, no 
reductionist fallacy, parsimony. The syntax chosen for comparison of the Arizona 
Department of Education’s alternative school accountability model and the Arizona 
State Board for Charter Schools’ Academic Framework reflects intended structure 


and a parsimonious presentation of the relationship among indicators and measures in 



6 


both frameworks. 


Methods, techniques, & mode of inquiry 

This study begins with descriptive research of the two state accountability 
frameworks and includes two years, fiscal year 2012 and fiscal year 2013, of school 
labeling with each framework. The sample is a census of all alternative schools that 
were labeled by each academic accountability framework in 2012 and in 2013. 

Then, analysis of the findings from this research suggests implications of these policy 
innovations. 


Data sources & evidence 

Data sources are publicly available documents from the Arizona Department of Education 
and the Arizona State Board for Charter School’s website. The websites include: 

• Department of Education presentations about the A - F models (ADE, 2012 & 2013) 

• Annual technical manual of the state’s labeling system (ADE, 2013 a & b) 

• Annual reports of Letter Grade labeling (ADE, 2012 & 2013) 

• The Arizona State Board for Charter Schools’ strategic plan (ASBCS, 201 1) 

• The Arizona State Board for Charter Schools’ Academic Framework (ASBCS, 2014) 

Also, the Arizona State Board of Charter Schools supplied its source files with calculations of 
Academic Dashboards. In addition, statute creating alternative education and regarding alternative 
school accountability is found in the Arizona Revised Statutes, Title 15, (LexisNexis, 2012). 
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Findings and Substantiated Conclusions 
State Context for Accountability of Alternative Schools 

This following timeline presents key statute and regulation regarding alternative school 
accountability in Arizona. 

1986 - Arizona Revised Statute §15-796 established alternative education. 

2000 - Arizona Revised Statute §15-241. J established a “parallel achievement profile for 
accommodation schools, alternative schools as defined by the state board of education...” 

2002 - State board of education defined alternative schools. The state adopted five categories for 
students served: 

1. Students with behavioral issues (documented history of disruptive behavior) 

2. Students identified as dropouts 

3. Students in poor academic standing or have demonstrated a pattern of failing grades 

4. Pregnant and/or parenting students 

5. Adjudicated youth (ADE, 2013) 

2004 - Arizona Department of Education published achievement profiles for alternative schools 
for the first time. 

2010 - Arizona legislature revised A.R.S. §15-241to create an A - F accountability system. 

201 1 - State Board of Education adopted A - F accountability system. The traditional school 
model was established, but not the parallel accountability model for alternative schools. 

2012 - State Board of Education approved policy specific to alternative schools. Alternative A - 
F was adopted and implemented for first academic year. 

2013 - Arizona State Board for Charter Schools approved an Academic Performance Framework 
specifically for alternative charter schools. 
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2014 - The Arizona State Board for Charter Schools amended its Academic Performance 
Framework and Guidance document to include Appendix G, a methodology section (ASBCS, 
2014). 

Arizona’s Alternative Schools 

Arizona has almost 2000, 1968 in 2012 and 1962 in 2013, public schools. Just less than 
10% of the schools are alternative schools. Alternative schools serve close to 30,000 students. It 
is difficult to report exact enrollment because most alternative schools have open enrollment, 
even to the end of the school year, and enrollment figures are a “snapshot” from a specific date. 
Arizona has roughly a million (1,023, 905) public school students (Aportela & Lackzo-Kerr, 
2013). Alternative schools serve about 3% of the public school population. Average number of 
students at an alternative school was 168 in fiscal year 2012 (Giovannone, 2012). Alternative 
schools are accommodation district schools, charter schools, and district schools. 


Table 1 reports the number and percentage of alternative schools in Arizona for fiscal 
years 2012 and 2013. 


Table 1 

Arizona’s Public Schools and Alternative Schools by Number and Percentage in 2012 and 2013 

Year Public Schools Alternative Schools 


2012 

1968 

165/1968, 8% 

2013 

1962 

184/1962. 9% 


About a quarter of Arizona’s public schools are charter schools. Alternative charter schools are between a 
fifth and a quarter of Arizona’s charter schools. 
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Table 2 


Year 

V/IIflUC1 ov-uuuia tmuiidinc Ltcnuuu, 

Charter Schools 

IHUllUVl U11U X VI ^rfVJX^rf UI1U 

Alternative Charter Schools 

2012 

2013 

521/1968, 26% of public schools 
519/1962, 26% of public schools 

102/521, 20% of charter schools 
118/519, 26% of charter schools 

Alternative charter schools are about two-thirds of Arizona’s alternative schools. 

Table 3 

Arizona’s Alternative Charter Schools bv Number and Percentage in 2012 and 2013 

Year 

Alternative Charter Schools 


2012 

2013 

102/165, 62% of alternative schools 
118/184, 64% of alternative schools 



Figure 1 presents a Venn diagram of Arizona’s public schools and the subsets of alternative 
schools, charter schools, and alternative charter schools. 


Public 



FIGURE 1 . Venn diagram of Arizona ’s public schools, public charter schools, and alternative 


schools 
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Arizona Department of Education’s Accountability Model for Alternative Schools 

The Arizona Department of Education developed a parallel accountability model for 
alternative schools early in 2012. The State Board of Education approved the model in May 
2012. Figure B presents the indicators and measures used in Arizona’s A-F Alternative model. 
Information about the model from the Technical Manuals (ADE, 2013 a & b) and accountability 
presentations (ADE, 2012 & 2013) explains the indicators and measures, plus data parameters 
such as Full Academic Year student counts, used in calculations. 


Growth Score 
(SGP + Improvement) 
70% weight 


Academic Outcomes 
30% weight 



ELL Reclassification 
Graduation Rate* 
Academic Persistence 


HS only 


Academic Outcomes + Growth Score + Additional Points = A-F ALT Letter Grade 


FIGURE 2. Arizona ’s A-F accountability model for alternative schools 
Source: Arizona Department of Education, 2012 A-F Technical Manual 

Arizona’s alternative school A-F Letter Grade model has two basic indicators, Academic 
Outcomes and Growth. Academic Outcomes is proficiency on the state-mandated assessment, 
Arizona’s Instrument to Measure Standards (AIMS) in mathematics and reading. The second 
indicator, Growth, has two measures, Student Growth Percentile (SGP) and Improvement. The 
Arizona Growth model, based on the work of Damian Betebenner (201 1) with its SGP is 
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explained in the Arizona A-F Letter Grade Technical manuals (2013 a & b). Improvement is an 
innovative measure of growth specific to the Arizona alternative school model. SGP can only be 
used to demonstrate growth through the 10 th grade, and the vast majority of testers at Arizona 
alternative high schools are re-testers. Students that are in 1 1 th grade or in a 4 th , 5 th , 6 th , 7 th , 
sometimes even 8 th year of being a “senior” have two opportunities each school year to test 
AIMS. Improvement measures the percentage of students who increased a performance band 
level on AIMS testing. AIMS uses four performance bands, “Fars Fall Below,” “Approaches,” 
“Meets,” and “Exceeds.” Student “improve” when they increase from one performance band to 
the next. Even if a student doesn’t “pass” the AIMS test, the student can “improve” by moving 
from “Fars Fall Below” to “Approaches.” The ADE Technical Manuals (2013 a & b) provide 
further explanation of “Improvement”. Academic Outcomes (proficiency) is weighted 30%. 
Each measure within Growth, SGP & Improvement, is weighted 35% for a total of 70%. 

In addition to the two basic indicators, Proficiency and Growth, and the three measures, 
AIMS proficiency, SGP, and Improvement, there are three components in which schools can 
receive additional points. The three additional point components are 

• ELL reclassification. This is the same formula used in Arizona’s traditional school 
model. 

• Graduation rate. 5 th year cohort-based graduation rate is looked at in terms of the 
graduation rate for all alternative high schools in Arizona. There are three ways for 
alternative high schools to earn these points. 

• Academic Persistence. This is another innovative component of the alternative school 
model. Academic Persistence additional points reward alternative schools for keeping 
students enrolled in public school. An academically persistent student is enrolled in an 
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alternative school at the end of one school year and then is re-enrolled in any public 
school the following year. 

Again, the Technical Manuals provide a more detailed explanation of the additional point 
components (ADE, 2013 a & b). 

ADE’s Alt A-F is a 200 point scale. The additional points are worth 3 points each for a 
total of 9 points. 3 out of 200 points is .015. Any additional points are added after the 70/30 
weighting for Growth and Proficiency is calculated. 

Arizona State Board for Charter Schools’ Academic Framework for Alternative Schools 

The Arizona State Board for Charter Schools uses the Arizona Department of Education 
designation of alternative school status to identify its alternative charter schools. Around two- 
thirds (2/3) of Arizona’s alternative schools are charter schools and about a bit a fifth of 
Arizona’s charter schools are alternative charter schools. In the fall of 2012, just a few months 
after Alt A-F was first calculated and publicly released in August 2012, the Arizona State Board 
for Charter Schools (ASBCS) developed and approved an Academic Framework for alternative 
charter schools. The Academic Performance Framework and Guidance document presents the 
alternative charter school framework, the weighting of indicators and measures, and the 
methodology (ASBCS, 2014). The ASBCS Academic Framework uses data collected by and 
analyzed by the Arizona Department of Education. Most of the components of the Academic 
Framework are the same as those found in the Arizona Department of Education’s alternative 
school model, yet Improvement and Academic Persistence are calculated a bit differently, plus 
there are sub-group measures for proficiency. The Academic Framework weights the indicators 
and measures differently and includes the school’s Alt A-F letter grade as one indicator. 
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The indicators for the ASBCS Academic Framework for alternative schools are: 

1. Student Progress over Time, or Growth 

2. Student Achievement, or Proficiency 

3. A-F Letter Grade State Accountability 

4. Post-Secondary Readiness (for high schools only) 

There are two measures of Growth, SGP and SGP for the bottom 25% of students at 
elementary and middle alternative schools and Improvement on AIMS performance band for 
high schools. Proficiency has two measures, the overall percent passing and subgroup 
proficiency. The Framework looks at proficiency for three sub-groups - English Language 
Learners, Lree & Reduced Lunch eligible students, and special education students. Sub-group 
proficiency allows a closer look at how the charter school serves these distinct student 
populations. The measure of the A-L Letter Grade State Accountability System is the letter 
grade the school received. Alternative high schools are evaluated for student Post-Secondary 
Readiness by High School Graduation Rate, as measured in the ADE Alternative School model, 
and Academic Persistence. 

Ligure 3 shows the Lramework’s measures of alternative charter school accountability 


and their weighting. 
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Alternative Charter Schools Weight 

Measure 

Elementary 
and Middle 

High 

School 

K-12* 

la. SGP 

30% 

5% 

15% 

lb. SGP of Bottom 
25% (Improvement 
for alternative high 
schools) 

20% 

25% 

** 

25% 

2a. Percent Passing 

15% 

20% 

15% 

2c. Subgroup 
proficiency 

10% 

10% 

10% 

3a. A-F Letter 
Grade State 
Accountability 
System 

10% 

5% 

5% 

4a. High School 
Graduation Rate 

NA 

15% 

15% 

4b. Academic 
Persistence - 
(Alternative 
Schools) 

15% 

20% 

15% 


This category includes any grade ranges across K-12 that do not fall solely in K-8 or 9-12. 

An Alternative K-12 School will be evaluated for both "SGP of Bottom 25%" for its Elementary and Middle 
School Students and "Improvement" for its High School Students. The 25% weighting will be divided equally 
between the two measures and within each measure divided equally between math and reading. 

FIGURE 3. Measures and weighting in Arizona State Board for Charter Schools ’ Academic 
Framework for Alternative Schools 


Each year after public issue of letter grades in August and data is available from the state’s 
student assessment information system for calculation by Research and Evaluation at Arizona 
Department of Education, the Arizona State Board for Charter Schools issues an Academic 
Dashboard for each of its authorized alternative charter schools. There are a few alternative 
charter schools that are authorized by other entities and do not receive Academic Dashboards. 
Figure 4 illustrates an academic dashboard for an alternative charter high school that was 
designated as an alternative school in both 2012 and 2013. 
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2012 

Alternative 

High School (9-12) 

2013 

Alternative 
High School (9-12) 

1 . Growth 

Measure 

Points 

Assigned 

Weight 

Measure 

Points 

Assigned 

Weight 

la. SGP 

Math 

33 75 

2.5 

NR 

0 

0 


2.5 

NR 

0 

0 

1b. Improvement 

Math 

24 50 

12.5 

25.3 50 

15 

Reading 

33 50 

12.5 

50 

75 

15 

2. Proficiency 

Measure 

Points 

Assigned 

Weight 

Measure 

Points 

Assigned 

Weight 

2a. Percent Passing 

Math 

19 / 19.6 50 

10 

24 / 19.1 

75 

10 

Reading 

50 / 47.3 75 

10 

58 / 52.1 75 

10 

2b. Subgroup ELL 

Math 

NR 

0 

0 

12/21.2 50 

1.67 

Reading 

NR 

0 

0 

54 / 48.2 75 

1.67 

2b. Subgroup FRL 

Math 

18 / 18.5 50 

2.5 

24 / 18.2 75 

1.67 

Reading 

46 / 46.1 50 

2.5 

59 / 50.4 75 

1.67 

2b. Subgroup SPED 

Math 

14 / 4.8 75 

2.5 

5 / 5.3 50 

1.67 

Reading 

21 / 20.2 75 

2.5 

6/21.6 50 

1.67 

3. State Accountability 

Measure 

Points 

Assigned 

Weight 

Measure 

Points 

Assigned 

Weight 

3a. State Accountability 

C-ALT 50 

5 

C-ALT 50 

5 

4. Graduation 

Measure 

Points 

Assigned 

Weight 

Measure 

Points 

Assigned 

Weight 

4a. Graduation 

Not Met 50 

15 

Not Met 50 

15 

4b. Academic Persistence 

67 50 

20 

87 75 

20 

Overall Rating 

Overall Rating 


Overall Rating 


Scoring for Overall Rating 

89 or higher: Exceeds Standard 

<89, but > or = to 63: Meets Standard 

<63, but > or = to 39: Does Not Meet Standard 

Less than 39: Falls Far Below Standard 

53.75 

100 

65 

100 


FIGURE 4. Illustration of an Arizona State Board for Charter Schools' academic dashboard for 
an alternative charter school in 2012 and 2013 


Comparison of the Two Accountability Models 

The Arizona State Board for Charter Schools’ Academic Performance Framework and 
Guidance document states, “The academic framework includes measures that are similar to 
components of the Arizona A-F Fetter Grade Accountability System. . (2014, p.4). Some 
measures are calculated identically, yet there are some differences. Certain technical nuances of 
the calculations are explained in the ADE Technical Manuals (2013a & b) and the Methodology 
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appendix of the ASBCS Guidance document (2014.) It is not the purpose of this paper to 
reproduce what has already been written, but rather to provide an overview comparison. 

Growth 


Student Growth Percentile 

ADE calculates SGP for alternative schools by pooling three years of data. In 
2012, “To be included in the pooled measure, the SGP must have been associated to a student 
who was FAY (Full Academic Year - my clarification) in the current year. (ADE, 2013a, p. 36) 
In 2013, ADE modified its pooling method. A school could lack grade 10 students with a 
current year (FY 13) SGP entirely yet still have SGP data for the alternative school letter grade 
as a result of students with SGP in the two prior years (ADE, 2013b) In contrast, ASBCS does 
not use three year pooling for SGP. That Framework does, however, require a minimum of 10 
students with data to be rated for that current year on any measure. The ASBCS framework uses 
a bottom 25% SGP of enrolled students at an elementary or middle alternative charter school as a 
measure of its Growth indicator. ADE’s Alt A-F does not. 

Improvement 

ADE calculates Improvement for elementary and middle alternative schools. ASBCS 
does not. ADE’s calculation of Improvement includes valid test records from the two most 
recent test administrations at any Arizona school. In the ASBCS Academic Framework’s 
calculation for high school students, students must be enrolled at the same school for both of the 
compared assessments. Improvement in the Academic Framework measures the percentage of 
non-proficient high school students who improve at least one performance level. In the state 
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metric, students at the “Meets Standard” AIMS performance level are given the opportunity to 
move to “Exceeds Standard (ASBCS, 2014). 

Proficiency 

Both the Alt A-F model and the Academic Framework include Proficiency, percent 
passing the AIMS. Both use the “better of’ test scores for high school students because students 
are given the opportunity to re-test after their first attempt to pass as 10 th graders in the Spring 
test administration. ADE combines the Math and Reading percent passing into one score. In 
contrast, the ASBCS Framework splits out Math and Reading results. Further, the ASBCS 
Academic Framework includes results for sub-groups; English Fanguage Feamers, Free and 
Reduced Funch eligible students, and students receiving Special Education. 

Graduation Rate and Academic Persistence 

Both Academic Persistence and Graduation Rate are additional points in the ADE Alt A- 
F model. The Arizona State Board for Charter Schools uses the ADE calculation for Graduation 
Rate, yet places it within the Framework and weights it differently. ASBCS calculates Academic 
Persistence slightly differently, then places it within the Framework and weights it differently. 
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ADE's 
Alt A - F 

Indicator 

Measure 

ASBCS 

Academic Framework for Alt Schools 

(minimum of 10 for rating per measure) 

30% 

Proficiency 

30% 

30% 

Simple average of Math & 
Reading 

% passing state- 
mandated exam 
(AIMS) 

20% 

Math and Reading 

10% Subgroups 

English Language Learner, Free & Reduced Lunch, 
Special Education 

70% 

(SGP + Improvement) 

Growth 

30% 

(SGP & Improvement) 

35% 

SGP 

30% - Elementary & Middle Schools 
15% - K-12 Schools 
5% - High Schools 

20% SGP of students in Bottom 25% of students - 
Elementary & Middle Schools 
25% for K-12 schools 

35% 

All grade levels 

Improvement 

25% 

high school only 

NA 

A-F Letter Grade 

5% 

(Not Component of FY12 & 
13 models) 

College & Career 
Readiness 

35% 

additional points 

HS Graduation Rate 

15% 

additional points 

Academic Persistence 

20% 

additional points** 

ELL Reclassification 

not included 


W 


An Alternative K-12 School will be evaluated for both "SGP of Bottom 25%" for its Elementary and Middle 
School Students and "Improvement" for its High School Students. The 25% weighting will be divided equally 
between the two measures and within each measure divided equally between math and reading. 

** Minimum of 10 students 

FIGURE 5. A comparison of Arizona’ s two accountability systems for alternative schools 
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Results of Alternative School Labeling 

Arizona Department of Education’s Alternative A-F Labels 

ADE issued Alt A-F labels for FY 12 and FY 13. The bar graphs in Figures 6 and 7 show 
the labels in terms of number of schools labeled and percentage. 


Alternative A-F Letter Grade 




A-ALT 

B-ALT 

C-ALT 

D-ALT 

Not Rated 

m FY 12 

9 

37 

51 

33 

35 

■ FY 13 

9 

34 

51 

51 

00 

m 


FY 12 ■ FY 13 


FIGURE 6. 2012 and 2013ADE Alternative A-F letter grades by number of schools 


Alternative A-F Letter Grade 



A-ALT 

B-ALT 

C-ALT 

D-ALT 

Not Rated 

■ FY 12 

5% 

22% 

31% 

20% 

21% 

21% 

■ FY 13 

5% 

18% 

28% 

28% 


m FY 12 ■ FY 13 


FIGURE 7. 2012 and 2013ADE Alternative A-F letter grades by percentage of alternative 
schools 
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ADE published letter grades for 165 alternative schools in 2012 and 184 in 2013. Several 
factors including school closures, alternative schools becoming an alternative education program 
at another district school, and schools converting from traditional to alternative status contributed 
to the difference in number of alternative schools between those two years. Alt A-F uses a 
distribution scale when applying letter grades. ADE calculates the points earned by all 
alternative schools in a certain year. Schools that had points at or above one and one-half 
standard deviation above the mean for alternative schools in that year were labeled A- Alt, 
schools with scores between one-half (Vi) and one and one-half (1 Vi) standard deviation above 
the mean were labeled B-Alt, C-Alts were one half of a standard deviation above and below the 
mean, alternative schools one-half (Vi) a standard deviation below the mean or lower were 
labeled D-Alt. Figure 8 from the 2013 ADE Technical Manual (p. 39) illustrates the distribution 
scale. (Please note that there is a typo on the B-Alt arrow.) 


100 % 

90% 

80% 

70% 

60% 

50% 

40% 

30% 

20 % 

10 % 

0 % 


B-Alt 


C-Alt 


D-Alt 


<- + 1 'ASD 


-'ASD 


Mean 


•-'ASD 


FIGURE 8. Distribution scale used for alternative schools by Arizona Department of Education 
Source: Arizona Department of Education 's 2013 Technical Manual 
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As ADE states in its 2013 Technical Manual, “Letter grades issued in 2012 also utilized this distribution 
scale, therefore, the approximate percentage of A- ALT, B-ALT, C-ALT, and D-ALT schools remained 
relatively consistent from 2012 to 2013” (p. 39). 

The “cut points” for alternative school letter grades changed from 2012 to 2013 because the mean 
for all alternative schools increased from 121 points in 2012 to 125 points in 2013. The standard 
deviation remained almost the same: 30.5 in 2012 and 31.2 in 2013. Table 4 presents the scales. 

Table 4 

Arizona Department of Education’s Alternative School Distribution Scale, 2012 and 2013 

Year A-Alt B-Alt C-Alt D-Alt 


2012 

200-167 

166-132 

131-97 

96-0 

2013 

200-171 

170-140 

139-109 

108-0 


The Arizona State Board for Charter Schools has issued 2012 and 2013 academic 
performance dashboards for the alternative charter schools that it authorizes. Again, bar graphs 
present results, both by number and percentage. 
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Alternative Charter School Academic 
Framework Ratings: FY 12 & 13 


sd= 



Not Rated 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

■ FY 12 

4 

0 

33 

57 

4 

■ FY 13 

2 1 

21 

92 

2 


FIGURE 9. 2012 and 2013 academic rating for alternative charters by number of schools 


Alternative Charter School Academic 
Framework Ratings: FY 12 & 13 


. I 



Not Rated 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

■ FY 12 

4% 

0% 

34% 

58% 

4% 

■ FY 13 

2% 

1% 

18% 

78% 

2% 



FIGURE 10. 2012 and 2013 academic rating for alternative charters by percentage of schools 
Findings show that schools are labeled differently by these two alternative schools 
accountability frameworks. In 2013, 80% of the alternative charter schools meet or exceeded 
the Academic Framework for Alternative Schools while less than a quarter of alternative schools 


received Alt A’s and Alt B’s. 
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A correlation coefficient was calculated for each year, 2012 and 2013, of these labels. 


Correlation of ASBCS and A-F Scores: FY12 
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FIGURE 11. Scatterplot of 2012 Correlation of Arizona Department of Education 1 s Alternative 
A-F and Arizona State Board for Charter Schools Academic Framework 


Correlation of ASBCS and A-F Scores: FY13 
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FIGURE 12. Scatterplot of 2013 correlation of Arizona Department of Education' s Alternative 
A-F and Arizona State Board for Charter Schools Academic Framework 
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The Pearson r for 2012 is 0.68 and 2013 is 0.78. Correlations are strong between the two 
accountability labeling systems. 

Substantiated Conclusions 

The impact of labeling using a distribution scale may adversely affect decisions about 
school quality when an A-Alt or B-Alt label is considered to be the indication of a “high quality” 
school. Only alternative schools one-half (1/2) a standard deviation or more above the mean for 
all Arizona alternative schools are labeled with an A- Alt or Alt-B by ADE’s letter grading 
system. Does that mean that only about a quarter of Arizona’s alternative schools are “quality” 
schools, or is this more a statistical consequence of using a distribution scale? In contrast, the 
ASBCS Academic Framework for Alternative Schools identifies expectations for alternative 
schools. Even though “Meeting Expectations” is relative to the performance of all Arizona 
alternative schools including accommodation district schools and district alternative schools, on 
some measures, e.g. percent passing, SGP, targets are set for other measures, e.g. Improvement, 
Academic Persistence, and Graduation. In 2013, the difference between the percentages of 
charter alternative schools that “met & exceeded” the Framework, eighty (80) percent, is a stark 
contrast to the twenty-three (23) percent of all Arizona alternative schools that are labeled A-Alt 
and B-Alt. In that same year, there were 1 19 alternative charter schools of the 184 alternative 
schools in Arizona, almost 2/3. Most of those alternative schools “met” the Academic 
Framework, yet the majority was labeled “C-Alt.” 

Discussion 

Contributing factors to the contrast between the two accountability systems include the 
weighting in the two models and the more detailed analysis found in the AZ State Charter School 
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Board’s Academic Framework. 

The innovative measures of alternative school accountability, Improvement and 
Academic Persistence, illustrate the impact that weighting can have on school labels. 
Improvement is one-half, 35%, of the Growth component in Alt A-F; however, as stated in 
ADE’s 2013 Technical Manual, “When no students at the school were eligible for improvement, 
the growth scored was made up entirely by the pooled SGP measure” (p.35). It would be 
possible for a school to not have current year SGP, but still have a pooled SGP score for a few 
students in the two previous years. Such a school would still have a Growth component score in 
Alt A-F, and Growth is 70% of the model. Does that result in appropriate current year labeling 
for that alternative school? Further, since Alt A-F used a distribution scale, the letter grades for 
all alternative schools in the state could be impacted. 

In contrast, the AZ State Board for Charter School’s Academic Framework does not use 
Improvement as a measure for elementary and middle schools. Improvement is a measure of the 
Growth indicator for high schools and weighted 25%. The expectation targets for “meeting” this 
measure are set and do not change from year to year. 

Academic Persistence is an innovative measure used in both academic accountability 
systems. Academic Persistence is three (3) additional points in the Alt A-F 200 point model. In 
contrast, Academic Persistence is 15% for elementary, middle, or K-12 alternative charter 
schools and 20% for alternative charter high schools evaluated by the AZ State Board for Charter 
Schools Academic Framework. As a policy-value decision, the Arizona State Board of 
Education should revisit the placement of Academic Persistence in the alternative school model. 
Alternative schools are making a valuable contribution to Arizona society by providing students 
with an education that encourages them to remain enrolled in a public school. Alternative 
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schools with academically persistent students may be one of the solutions to the state’s and the 
U.S.’s dropout problem. 


Upcoming Changes 

Multiple changes are coming to alternative school accountability in Arizona. In late 
February 2014, Arizona’s State Board of Education approved a revamped alternative school 
definition and process for alternative school status. Outcome data on the impact of the revised 
definition and process will not be available until the summer of 2014. That will be an area for 
further research. In addition, the Arizona Department of Education is proposing revisions to the 
Alt A-F model such as moving graduation rate inside the model as part of a College and Career 
Readiness Index. Such action will set precedent for moving a component that currently is 
designated as “additional points” to a component within the model. Just as graduation rate, in 
some form, will become a weighted component within Alt A-F, Academic Persistence should be 
a component within the model. Alternative schools are advocating that the Arizona State Board 
of Education set some targets within the model, rather than using a distribution scale for the 
entire model each year. When the Arizona Department of Education changes certain components 
of its Alt A-F model, some measures within the Arizona State Board for Charter Schools’ 
Academic Framework will need revision. There will be a need for additional research on the 
new configuration of both models in the upcoming years. 

Scholarly Significance of the Study 

This study furthers the limited research currently available for making informed policy 
decisions on alternative education accountability. Appropriate accountability for alternative 
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education campuses is currently being discussed in various forums (National Alliance for Public 
Charter Schools, 2013: National Association for Charter School Authorizes, 2011 & 2013). The 
Center for Student Achievement has called for more implementation of the innovative measures, 
Academic Persistence and Improvement, in all of Arizona’s school accountability models 
(Aportela & Lackzo-Kerr, 2013). This inquiry furthers the limited body of research and 
scholarship about alternative school accountability in the United States with its analysis of 
Arizona’s alternative school accountability systems, is an attempt to clarify the two 
accountabilities through comparison and contrast, and suggests some areas for further research and 
evidence-based policy decisions regarding more appropriate accountability for alternative schools. 
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Addendum 


During the session at the American Educational Research Association meeting, the chair 
asked, ‘Can you show a break-down of how the alternative schools that received both “labels,” 
the Department of Education’s A - F and the State Board for Charter Schools’ Academic 
Framework, fared?’ 


The figures below present that data for both fiscal years. Please note that the percentages 
are percentages of alternative charter schools that received both (my emphasis) labels. These 
numbers and percentages are not the same as the numbers and percentages presented in Figures 9 
and 10 because this is a subset of the schools reported in those figures. 



AZ State BD Charter Schools' Academic Framework 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

A-ALT 



4 

4 

B-ALT 


3 

25 


C-ALT 


18 

19 


D-ALT 


11 

4 





AZ State BD Charter Schools' Academic Framework 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

A-ALT 

0% 

0% 

4% 

4% 

B-ALT 

0% 

3% 

28% 

0% 

C-ALT 

0% 

20% 

21% 

0% 

D-ALT 

0% 

12% 

4% 

0% 


FIGURE 13. Alternative charter schools with both labels, an Arizona Department of Education 
letter grade and Arizona State Board for Charter Schools’ academic framework rating, in fiscal 
year 2012, by numbers and percentages 
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AZ State BD Charter Schools' Academic Framework 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

A-ALT 



5 

2 

B-ALT 



28 


C-ALT 


1 

41 


D-ALT 

1 

17 

13 





AZ State BD Charter Schools' Academic Framework 

Falls Far Below 

Does Not Meet 

Meets 

Exceeds 

A-ALT 

0% 

0% 

5% 

2% 

B-ALT 

0% 

0% 

26% 

0% 

C-ALT 

0% 

1% 

38% 

0% 

D-ALT 

1% 

16% 

12% 

0% 


FIGURE 14. Alternative charter schools receiving both labels, an Arizona Department of 
Education letter grade and Arizona State Board for Charter Schools ’ academic framework 
rating, in fiscal year 2013 by numbers and percentages 


